Hidden Evidence Behind Useless Replications

Experiments that are run with few experimental subjects are often considered not to be very reliable and deemed, as a result, to be useless with a view to generating new knowledge. This belief is not, however, entirely correct. Today we have tools, such as metaanalysis, that we can use to aggregate small-scale experiments and output results that are equivalent to experiments run on large samples that are therefore reliable. The application of metaanalysis can overcome some of the obstacles that we come up against when running software engineering experiments (such as, for example, the practitioner availability problem).